Testing Optimality of Sequential Decision-Making

This paper provides a statistical method to test whether a system that performs a binary sequential hypothesis test is optimal in the sense of minimizing the average decision times while taking decisions with given reliabilities. The proposed method requires samples of the decision times, the decision outcomes, and the true hypotheses, but does not require knowledge on the statistics of the observations or the properties of the decision-making system. The method is based on fluctuation relations for decision time distributions which are proved for sequential probability ratio tests. These relations follow from the martingale property of probability ratios and hold under fairly general conditions. We illustrate these tests with numerical experiments and discuss potential applications. This work has been partly supported by the German Research Foundation (DFG) within the Cluster of Excellence EXC 1056 ’Center for Advancing Electronics Dresden (cfaed)’ and within the CRC 912 ’Highly Adaptive Energy-Efficient Computing (HAEC)’. The material in this paper has been presented in part at the IEEE International Symposium on Information Theory (ISIT), Aachen, Germany, June 2017 [1]. M. Dörpinghaus is with the Vodafone Chair Mobile Communications Systems and with the Center for Advancing Electronics Dresden (cfaed), Technische Universität Dresden, 01062 Dresden, Germany (e-mail: [email protected]). Izaak Neri is with the Max-Planck-Institute for the Physics of Complex Systems, Dresden, Germany, with the Max-PlanckInstitute of Molecular Cell Biology and Genetics, Dresden, Germany, and with the Center for Advancing Electronics Dresden (cfaed), Technische Universität Dresden, 01062 Dresden, Germany (e-mail: [email protected]). Édgar Roldán is with the Max-Planck-Institute for the Physics of Complex Systems, Dresden, Germany, with the Center for Advancing Electronics Dresden (cfaed), Technische Universität Dresden, 01062 Dresden, Germany, and with GISC – Grupo Interdisciplinar de Sistemas Complejos, Madrid, Spain (e-mail: [email protected]). H. Meyr is an emeritus of the Institute for Integrated Signal Processing Systems, RWTH Aachen University, 52056 Aachen, Germany and is now a grand professor of the Center for Advancing Electronics Dresden (cfaed) at Technische Universität Dresden, 01062 Dresden, Germany (e-mail: [email protected]). Frank Jülicher is with the Max-Planck-Institute for the Physics of Complex Systems, Dresden and with the Center for Advancing Electronics Dresden (cfaed), Technische Universität Dresden, 01062 Dresden, Germany, Germany (e-mail: [email protected]). January 8, 2018 DRAFT ar X iv :1 80 1. 01 57 4v 1 [ cs .I T ] 4 J an 2 01 8